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IMAGING SYSTEM AND METHOD FOR MONITORING AN EYE 

Technical Field 

[0001] The present invention generally relates to eye monitoring and, 

more particularly, relates to monitoring an eye, such as one or both eyes of a 
driver of a vehicle, with a light illuminated video imaging system. 



Background of the Invention 

[0002] Video imaging systems have been proposed for use in vehicles 

to monitor the driver and/or passengers in the vehicle. Some proposed video 
imaging systems include one or two cameras focused on the driver of the 
vehicle to capture images of the driver's face and allow for determination of 
various facial characteristics of the driver including the position, orientation, 
and movement of the driver's eyes, face, and head. By knowing the driver 
facial characteristics, such as the driver's eye position and gaze, vehicle 
control systems can provide enhanced vehicle functions. 

[0003] A vehicle control system can monitor one or both eyes of the 

driver and determine a condition in which the driver appears to be drowsy, 
and can take further action to alert the driver of the driver drowsy condition. 
The vehicle control system can also determine if the driver is distracted, and 
take action to minimize the distraction. Further, the vehicle control system 
can monitor the driver eye gaze and control various vehicle systems, such as 
the radio and entertainment system, and provide enhanced control of such 
systems. 

[0004] Some proposed vision-based imaging systems that monitor the 

eye(s) of the driver of a vehicle require infrared (IR) illumination along with 
visible light filters to control scene brightness levels inside of the vehicle 
cockpit. The ability to discern facial characteristics is a considerable 
requirement in the case of driver monitoring. Current techniques for 
illuminating the driver's face with infrared illumination include providing a 
light source focused on the driver's eye(s). Other techniques for illuminating 
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the driver's face propose employing multiple light sources of different 
wavelengths, arranged in a generally concentric ring, for generating a bright 
eye pupil and dark eye pupil effect. 
[0005] Some proposed eye monitoring techniques process video images 

taken during both the bright eye pupil and dark eye pupil illuminations, and 
process the difference in the acquired bright and dark eye pupil images. By 
taking the difference in images acquired during bright and dark illumination, 
glare and other undesirable effects caused by bright images can be removed to 
allow for enhanced identification of the target eye. While combined dark and 
bright eye pupil illuminations have been proposed, such techniques typically 
require a large number of light sources of multiple wavelengths arranged in 
the shape of a concentric ring adjacent to and surrounding the video imaging 
camera. 

[0006] It is therefore desirable to provide for an effective method for 

illuminating an eye and generating images of the eye to allow for enhanced 
eye monitoring of a person in the vehicle. In particular, it is desirable to 
provide for a more cost affordable system that offers flexible packaging of the 
light source illuminators and is easy and economical to manufacture. 

Summary of the Invention 
[0007] In accordance with the teachings of the present invention, a 

system and method are provided for monitoring an eye. According to one 
aspect of the present invention, the system includes a video imaging camera 
oriented to generate images of an eye. The camera is aligned along an 
imaging axis. A first light source is located at an angle less than 2.5 degrees 
from the imaging axis for illuminating the eye. A second light source is 
arranged at an angle greater than 4.5 degrees from the imaging axis for 
illuminating the eye. The video imaging camera acquires a first image when 
the first light source illuminates the eye, and generates a second image when 
the second light source illuminates the eye. The system further includes a 
processor for processing the first image and the second image. 



[0008] According to another aspect of the present invention, the 

method includes arranging a video imaging camera oriented along an imaging 
axis to generate images of an eye. The method also includes arranging a first 
light source at an angle less than 2.5 degrees from the imaging axis, and a 
second light source at an angle greater than 4.5 degrees from the imaging axis. 
The method also includes the steps of illuminating an eye with the first light 
source, and generating a first image of the eye while the eye is illuminated 
with the first light source. The method further includes the steps of 
illuminating the eye with the second light source, and generating a second 
image of the eye while the eye is illuminated with the second light source. 
Further, the method includes processing the first and second images. 

[0009] These and other features, advantages and objects of the present 

invention will be further understood and appreciated by those skilled in the art 
by reference to the following specification, claims and appended drawings. 



Brief Description of the Drawings 
[0010] The present invention will now be described, by way of 

example, with reference to the accompanying drawings, in which: 
[0011] FIG. 1 is a top view of an eye monitoring system having video 

imaging camera located in the cockpit of a vehicle and projecting towards the 

eyes of a driver; 

[0012] FIG. 2 is a side elevational view of the projection of the eye 

monitoring system towards the eye of the driver; 
[0013] FIG. 3 is a front view of the eye monitoring system focused on 

an eye of the driver; 
[0014] FIG. 4 is a block diagram illustrating the eye monitoring 

system; 

[0015] FIG. 5 illustrates the acquisition of a video image of the eye 

shown with bright illumination of the eye pupil; 
[0016] FIG. 6 illustrates the acquisition of a video image of the eye 

shown with dark illumination of the eye pupil; 
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[0017] FIG. 7 is a flow diagram illustrating a routine for controlling 

the illumination and acquisition of images with the eye monitoring system; and 

[0018] FIG. 8 is a flow diagram illustrating a method of applying the 

eye monitoring system for use in exemplary vehicle applications. 

Description of the Preferred Embodiment 

[0019] Referring now to FIG. 1, the passenger compartment (cockpit) 

12 of a vehicle 10 is generally shown equipped with a mono-camera driver eye 
monitoring system 20. The eye monitoring system 20 has a video imaging 
camera and first and second light illumination sources located within the dash 
14 and focused on the person (driver) 18 driving the vehicle 10 for 
illuminating the face of the driver 18 and generating images of the driver 18. 

[0020] The eye monitoring system 20 is shown mounted generally in a 

mid-region of the dash 14 in the front region of the vehicle cockpit 12. The 
eye monitoring system 20 may be mounted in any of a number of various 
locations within the vehicle 10 which allow for light illumination and 
acquisition of video images of one or both eyes of the driver 18 of the vehicle 
10, For example, the video imaging camera 20 may be mounted in the 
steering assembly 16, or mounted elsewhere in the dash 14, or may be 
mounted in the instrument cluster. While the eye monitoring system 20 is 
shown and described herein in coimection with a single video imaging camera, 
it should be appreciated that two or more video imaging cameras may be 
employed, without departing from the teachings of the present invention. 

[0021] The eye monitoring system 20 is mounted to the dash 14 such 

that the light illumination sources illuminate the eye(s) of the driver and the 
video imaging camera captures successive video image frames of the region 
where the driver 18 of the vehicle 10 is expected to be located during normal 
vehicle driving. More particularly, the video image captures the driver's face 
including one or both eyes 22 and the surrounding ocular features generally 
formed in the area referred to as the ocular adnexa. The acquired video 
images are then processed for tracking characteristics of one or both eyes 22 
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of the driver 18. Each video frame image may be processed to determine 
whether one or both eyes 22 of the driver 18 are in an open position or a 
closed position to determine a driver drowsiness condition. The video frame 
images may be processed to determine a driver distraction condition, an 
inattentive driver, or other conditions. 
[0022] Referring to FIG. 2, the eye monitoring system 20 is shown 

having a housing 36 and a video imaging camera 30 mounted to the housing 
36 and focused on an eye 22 of the driver 18. The video imaging camera 30 
is shown focused along an imaging axis 38 at an inclination angle ^ relative to 
the horizontal plane of the vehicle 10. The inclination angle (j> is within the 
range of zero to thirty degrees ( 0** to 30°). An inclination angle <t> in the 
range of zero to thirty degrees ( 0° to 30**) generally provides a clear view of 
the driver's ocular features including one or both eyes 22 and the pupil of the 
eyes 22, the superior and inferior eyelids, and the palpebral fissure space 
between the eyelids. 

[0023] As shown in FIGS. 2 and 3, the eye monitoring system 20 also 

includes a first light illumination source 32 shown as a first plurality of 
infrared (IR) light emitting diodes (LEDs) 32 A, 32B, and 32C. The eye 
monitoring system 20 further includes a second light illumination source 34 
shown as a second plurality of infrared light emitting diodes 34 A, 34B, and 34 
C. The LEDs 32A-32C and 34A-34C are mounted to housing 36 and are 
spaced on opposite sides of video imaging camera 30, according to the 
embodiment shown. Alternately, the LEDs 32A-32C and 34A-34C could be 
arranged on the same side of camera 30. 

[0024] The video imaging camera 30 is focused on one or both eyes 22 

of the driver of the vehicle. In the embodiment shown, video imaging camera 
30 is focused on one eye 22. The video imaging camera 30 is aligned along 
an unaging axis 38, which generally defines a centerline of the focus of the 
camera 30 onto the eye 22. Line 33 defines the centerline between the first 
light illumination source 32 and the eye 22, while line 35 defines the 
centerline between the second light illumination source 34 and the eye 22. 



[0025] The eye monitoring system 20 employs the first and second 

light illumination sources 32 and 34 for illuminating the target eye 22, such 
that the video imaging camera 30 is able to capture images of the eye 22 and 
its pupil in bright and dark illuminated conditions. The first plurality of IR 
LEDs 32A-32C of light source 32 are shown configured in a straight line to 
one side of the video imaging camera 30. The first plurality of IR LEDs 32A- 
32C are mounted close to the video imaging camera 30 such that the centerline 
33 of first light source 32 is at an angle 0^ less than 2.5 degrees from the 
imaging axis 38. According to another embodiment, the first light source 32 
is arranged at an angle 0^ less than 2.3 degrees from the imaging axis 38. By 
arranging the first light source 32 at angle 0^ less than 2.5 degrees from the 
imaging axis 38. the first light source 32 is substantially coaxial with the 
imaging axis 38 such that the resultant light illumination produces a bright eye 
image. 

[0026] The second plurality of IR LEDs 34 A-34C of light source 34 

are shown configured in a straight line on the opposite side of the video 
imaging camera 30. The second plurality of IR LEDs 34A-34C are mounted 
close to the video imaging camera 30 such that the centerline 35 of the second 
light source 34 is at an angle 0^ greater than 4.5 degrees from the imaging 
axis 38. According to a further embodiment, the second light source 34 is 
arranged at an angle 0^ greater than 4.6 degrees from the imaging axis 38. 
By arranging the second light source 34 at angle greater than 4.5 degrees 
from ttie imaging axis 38, the second light source 34 is substantially non- 
coaxial with the imaging axis 38 such that the resultant light illumination 
produces a dark eye image. 

[0027] The first and second light sources 32 and 34 may include any of 

a number of commercially available lighting sources. In one embodiment, the 
light sources 32 and 34 each generate a light source at substantially the same 
frequency. While each of the first and second light sources 32 and 34 are 
shown made up of three LEDs, it should be appreciated that any of a number 



of one or more light lighting devices may be employed for each of light 
sources 32 and 34, without departing from the teachings of the present 
invention. 

In the example shown, the first plurality of IR LEDs 32A-32C 
are spaced at a distance of less than twenty-five millimeters (25 mm) from the 
center of the video imaging camera 30. In this example, the second plurality 
of IR LEDs 34A-34C are located at a distance greater than fifty 
millimeters (50 mm) from the video imaging camera 30. The video imaging 
camera 30 light source 32 and 34 are displaced from the eye 22 by a distance 
X of about six hundred fifteen millimeters (615 mm) according to this 
example. 

The first and second light sources 32 and 34 are illuminated 
such that only one of the first and second light sources 32 and 34 is on at a 
time, according to one embodiment. The first plurality of LEDs 32A-32C of 
first light source 32 are turned on to illuminate the eye 22 to enable the video 
imaging camera 30 to capture a bright first image of the eye 22. With the first 
plurality of LEDs 32A-32C turned off, the second plurality of LEDs 34A-34C 
of light source 34 are then turned on to illuminate the eye 22 so as to allow the 
video imaging camera 30 to generate a dark second image of the eye 22. The 
first and second video images captured during bright and dark images of the 
eye are then processed to discern information about the eye 22 and/or 
condition of the driver 18. 

Referring to FIG. 4, the eye monitoring system 20 is further 
shown having the video imaging camera 30 and first and second light sources 
32 and 34 coupled to a vision processor 40 which, in turn, is coupled to 
countermeasure systems 60 via input/output (I/O) 58. The video imaging 
camera 30 may include a CCD/CMOS active-pixel digital image sensor 
mounted as a chip onto a circuit board. One example of a CMOS active-pixel 
digital unage sensor is Model No. PB-0330, commercially available from 
Photobit, which has a resolution of 640 H x 480 V. It should be appreciated 
that other cameras may be employed. 



8 

[0031] The vision processor 40 is shown having a frame grabber 46 for 

receiving the video output frames on line 52 generated by the video imaging 
camera 30. Vision processor 40 also includes a video processor 42 for 
processing the acquired video frames. The vision processor 40 includes 
memory 44, such as random access memory (RAM), read-only memory 
(ROM), and other memory as should be readily apparent to those skilled in the 
art. The vision processor 40 may be configured to perform one or more 
routines for controlling the actuation of first and second light illumination 
sources 32 and 34, controlling the video imaging camera 30, processing the 
acquired video images, and applying the processed information to vehicle 
control systems. 

[0032] The vision processor 40 has a camera control function via 

control RS-232 logic 56 which allows for control of the video imaging camera 
30 via camera control signals on line 54. Control of the video imaging camera 
30 may include automatic adjustment of the pointing orientation of the video 
imaging camera 30. For example, the video imaging camera 30 may be 
repositioned to focus on an identifiable feature, and may scan a region in 
search of an identifiable feature, including the driver's face and, more 
particularly, one or both eyes 22. Control may also include adjustment of 
focus and magnification as may be necessary to track an identifiable feature. 
Thus, the eye monitoring system 20 may automatically locate and track an 
identifiable feature, such as one or both of the driver's eyes 22. 

[0033] The vision processor 40 further includes a timing sync pulse 

generator 48 coupled to video processor 42. The timing sync pulse generator 
48 produces timing sync output signals on line 50 to control the activation of 
the first and second light illumination sources 32 and 34. That is, the timing 
sync output signals control activation of light sources 32 and 34 by 
sequentially turning light sources 32 and 34 on and off, so that only one light 
source is on at a time. The vision processor 40 monitors the timing sync 
output signals and processes the video output signals as images that were 
acquired either with the first lighting source 32 or second lighting source 34. 



[0034] Referring to FIGS. 5 and 6, acquired first and second images of 

the eye 22 are shown for both bright eye pupil and dark eye pupil 
illuminations. In FIG. 5, the first light source 32 is turned on, while the 
second light source 34 is turned off, to generate the bright eye pupil image 
shown. The timing sync pulse generator 48 controls the light illumination 
timing, while the bright video captures 62 are stored in memory frame 64B. 
The resulting coaxial illumination of the eye 22 causes a bright eye pupil 
effect, such that the iris 24 and its pupil are captured as bright images. The 
bright pupil reflection is a result of the physiological properties of the human 
eye. The IR illumination is reflected out through the pupil from the retina and 
visible at the centerline of the camera. 

[0035] In FIG, 6, the second light source 34 is turned on, while the 

first light source 32 is turned off, to generate the dark eye pupil image shown. 
The dark video captures 62 are stored in memory frame 64D. The resulting 
non-coaxial illumination of the eye 22 causes a dark eye pupil effect, such that 
the iris 24 and its pupil are captured as dark images. The dark pupil reflection 
is a result of the physiological properties of the human eye. The IR 
illumination is reflected back into the eye away from the retina and is not 
visible at the centerline of the camera. 

[0036] The captured bright and dark video images of eye 22 may be 

processed to determine whether the eye 22 is in an open or closed position, 
indicative of a drowsy driver condition, or may be evaluated to determine if 
the driver is inattentive or distracted. This can be achieved by monitoring the 
sclera 26 and/or the iris 24 of the eye 22 and determining when at least one of 
the sclera 26 and iris 24 are not visible in the video image due to complete 
covering of the eyelid to determine a driver drowsiness condition. Driver 
inattention or distraction can be determined by monitoring the position of the 
iris 24, as well as positioning of the surrounding facial characteristics. 
[0037] Referring to FIG 7, a control routine 70 is illustrated for 

controlling the first and second light illumination sources 32 and 34 and video 
imaging camera 30 of the eye monitoring system 20. In step 72, routine 70 
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captures frame 1 for setup. Routine 70 then proceeds to step 74 to 
synchronize the illuminator timing, and then turns on the LED bank 1 (32 A- 
32c) of the first light source 32 in step 76. 

With the first light source 32 turned on, routine 70 captures a 
frame 2 image to create a bright pupil effect image in step 78. In step 80, the 
LED bank 2 (34A-34C) of the second light 34 source is turned off in step 80. 
In step 82, routine 70 generates a timing/sync pulse. Next, in step 84, the 
LED bank two of the second light source 34 is turned on in step 84. With the 
second light source 34 turned on, routine 70 captures a frame 3 image to 
create the dark pupil effect image in step 86. Next, in step 88, routine 70 
turns the LED bank 2 of the second light source 34 off in step 88, and then 
generates a timing/sync pulse in step 90, before returning to step 76. 
Accordingly, control routine 70 controls the timing for activating the first and 
second light sources 32 and 34 and capturing images generated during 
illumination of the first source 32 and then the second light source 34. 

Referring to FIG. 8, a routine 100 is illustrated for applying the 
eye monitoring system 20 of the present invention in any one or more of 
various vehicle applications. Routine 100 begins at step 102 and proceeds to 
illuminate and acquire the driver's eye images in step 104. The driver's eye 
images are then processed in step 106, according to the present invention. 
The processed eye images may then be used in step 122 in a driver distraction 
application in which routine 100 checks whether a driver is distracted in step 
124. If the driver is determined to be distracted, routine 100 may initiate one 
or more countermeasures in step 126. The countermeasures may include 
blocking cell phone calls in step 126A, disabling a radio/infotainment system 
in step 126B, and/or initiating important driving cues in step 126C. 
I The processed eye images may also be used in a driver gaze 

application as shown in step 114. In the driver gaze application, routine 100 
determines a driver gaze in step 116. The determined driver gaze may be 
employed to adjust the following distance of the vehicle in step 118 and/or 
utilize appropriate driving cues in step 120. 
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[0041] The processed eye images may further be used in a driver 

drowsy application as shown in step 108. In the driver drowsy application, 
routine 100 determines if the driver is drowsy in decision step 110. If the 
driver is determined to be drowsy, routine 100 initiates one or more 
countermeasures in step 112. These countermeasures may include an audible 
warning in step 112A, initiating a haptic seat in step 112B, increasing vehicle 
following distance in step 112C, initiating cold air conditioning (A/C) in step 
112D, and/or activating peppermint olfactory stimulation in step 112E. 

[0042] Accordingly, the eye monitoring system 20 of the present 

invention advantageously illuminates and acquires bright and dark images of 
the eye to allow for easy monitoring of the driver of the vehicle. The eye 
monitoring system 20 of the present invention is economical and easy to 
manufacture in a single package, and easy to deploy in a vehicle. It should be 
appreciated that the eye monitoring systems may be employed in various 
applications employed both on and off a vehicle. 

[0043] It will be understood by those who practice the invention and 

those skilled in the art, that various modifications and improvements may be 
made to the invention without departing from the spirit of the disclosed concept. 
The scope of protection afforded is to be determined by the claims and by the 
breadth of interpretation allowed by law. 



